Describe the live record's keyset half and R-PAGE's, cover the token lists end to end, and regenerate the stack's artifacts - #1575

Refs #1165

This is the top of the first #1165 stack. It documents what the two layers below added, the keyset half of the live GitLab record at schema 5 and the keyset half of R-PAGE that reads it, adds the end-to-end scenario for the two service account token lists the first layer changed, and regenerates once every committed artifact the stack moves, which the layers below leave stale under the freshness deferral.

The documentation

The contributor pages described schema 4 and an R-PAGE that raises no keyset finding by design. In docs/development/cmd-utilities.md, the gen_api_live section now says what the walk records for each route (the paginator, the file and line quoted, the helpers it went through, the parameters the next page is asked by, the feature flags with their defaults), why a site says where GitLab can serve a keyset page rather than that it does, and what -check refuses of that half. The R-PAGE paragraph reads keyset_routes off the record's keyset field (53 routes, 39 of them offset too) rather than off the params (the 10 that declare per_page without page), and a new paragraph describes the keyset half: its four problems and their two grains, the keyset-not-followable declarations, and the figures against the committed record (202 actions offering page_token, 20 of them on keyset routes that publish next_page_token and were seen following it, 182 on routes GitLab pages by offset alone, three lists declared). docs/development/request-paths.md says the same in its R-PAGE bullet, static-analysis.md and the repository map in a line each, and the R-PAGE clause of CLAUDE.md names the rule and nothing more.

On the site, the output format page, in English and Spanish, says that the two service account token lists, where GitLab refuses every keyset request, no longer take pagination or page_token, refuse either before sending anything and page with page, and that the lists still answering a keyset request with 405 are the ones given an order GitLab cannot page by keyset.

The end-to-end scenario

Nothing drove either token list against a real GitLab with a keyset argument, or with a sort order at all. Two scenarios in test/e2e/gitlab/common/keyset_pagination_test.go, TestKeysetPagination_GroupServiceAccountTokens_PageByOffsetAlone and TestKeysetPagination_ProjectServiceAccountTokens_PageByOffsetAlone, give a service account three tokens named so they sort in the order they were made, and on every surface:

• send pagination: "keyset", page_token and order_by, each held to a refusal made before any request: the SDK's on the individual surface, whose schema admits no property it does not list, execute's Unknown params on the dynamic surface, and the meta dispatcher's unknown field;
• follow the list one token a page by page number in name_asc order, holding every page to carry no next_page_token and to name the next page while has_more holds;
• read it whole in name_desc order, so an order GitLab ignored, which leaves the tokens unsorted, reads as a failure rather than as a listing.

Both were red on the parent: driving the server built from e3e09d290, the base of the stack before it was rebased onto main, against CE 19.5.0, every surface failed at the first argument (8 tests, 8 failures), because that server still sends pagination=keyset and GitLab answers it, 405 {error: Keyset pagination is not yet available for this type of request}: refused as tool_error, and the test expected invalid_params on the individual surface and the error does not mention "unknown field \"pagination\"" on the other two. That run was of the scenarios' first version, whose dynamic case expected the meta wording; the complete CE run of this stack showed that execute refuses the argument earlier, with Unknown params: pagination. and the list of valid params, and the dynamic case now expects that. With the final version, the red run on the parent fails again on every surface: GitLab answers 405 {error: Keyset pagination is not yet available for this type of request} for both group.service_account_pat_list and project.service_account_pat_list. On the stack's head before the rebases, the green run of the two scenarios passes.

The regeneration

One commit, with the comparison checked:

• the tool snapshots of the individual surface and of the compact and full meta schemas, which change on exactly the two token list tools and the gitlab_group and gitlab_project schemas that carry them (pagination, page_token and order_by gone, sort naming the eight documented orders); the curated tools_meta.json does not change;
• the per-domain tool reference of the group and project groups, in English and Spanish, the three llms files, the token footprint with the README block it writes, and the testing reference, refreshed with -skip-coverage;
• the request inventory, recorded again on the whole unit suite and merged: the two token routes are no longer recorded sending order_by, page_token or pagination, and the group one is now recorded sending page and per_page.

The action request record and the fine-grained table, the site stats, the brand assets, the manifests and the e2e coverage page regenerate unchanged.

How I tested it

The stack was written on e3e09d290 and rebased twice: onto main at 6c9066aef, after the OpenSSF Best Practices stack, and onto main at 56758fb71, after the access level and edition tier stack (#1567 to #1571). In this layer's two hand-written commits the first rebase changed one place: main added rows to the pipe table of docs/development/static-analysis.md and padded it again, so I took main's table, put this layer's sentence back into the check-api-live row and ran go run ./cmd/format_md_tables/. The second changed nothing in them: what main changed in CLAUDE.md, docs/development/cmd-utilities.md and docs/development/repository-map.md lies outside the lines this layer edits there, and merged without a conflict. Both times the regeneration commit was not carried over: I dropped it and regenerated every artifact on the rebased tree. It changes the same 15 files as before. The tool snapshots, the tool reference, the llms files and the request inventory, recorded again on the whole rebased unit suite, change line for line as they did on the old base. The token footprint with its README block and the testing reference move by the same amounts from main's new figures: 225 fewer tokens on each tier's individual surface, and 37 more test functions and three more test files under cmd/, the stack's own tests.

With the freshness checks on, on this head after the second rebase: go test ./... -count=1 passes (301 packages), and again with GITLAB_MCP_TEST_SNAPSHOT_PARITY=deferred. make analyze passes 30 of its 32 steps on the remote runner, and the other two pass locally on the same tree: markdownlint (step 6, which needs Node) and the copyright and license headers (step 32, which reads git's file list). Every check the "Generated artifacts" job of CI runs passes, among them check-tool-reference, check-llms, check-footprint, check-site-stats, check-testing-docs, check-request-inventory, check-api-live, audit-1to1-paths, check-action-grants, check-e2e-static, check-tenancy and check-file-headers, with check-openplugin and check-server-json-packages run locally since they need the network. make check-dco passes for the ten commits of the stack (the tenth is the second layer's answer to SonarCloud's rubydre:S131, which changes no recorded output). make test-e2e-http (780 tests, 8 skipped because the runner has no Docker) and make test-e2e-stdio (277 tests) pass, the e2e packages compile, and test/e2e/gitlab/common passes golangci-lint run and golangci-lint fmt --diff. The site passes i18n:check, facts:check, chips:check, llms:check, build and lint, and node scripts/check-doc-links.mjs passes.

Against Docker GitLab 19.5.0, the complete CE run (make test-e2e-ce with the skip gate) ran on this stack before the dynamic case of the two new scenarios was corrected: 1313 tests, and the only failures were those two dynamic cases, for the wording above. Its skip gate also failed on four skips, the Bitbucket Cloud and GitHub imports, because that worktree had no .env with their credentials, which the next run has. The correction touches only that test file. On the stack's head before the rebases (25b20c981), with the skip gate on, the complete CE run passes 1313 tests (1 skip, which is declared) and the complete EE run passes 1642 tests (no skips).

I did not run the Docker suites again after the rebases. Since that run, the server code this stack changes is the same apart from the comment that names the register row (now 143) and the file headers, and the Ruby script the second layer's SonarCloud commit splits, which only gen_api_live runs, inside a GitLab it boots. What main brought that those suites exercise is the dependency update of #1564 (golang.org/x/net 0.62.0, google.golang.org/grpc 1.85.0, gqlparser 2.5.64 and tiktoken-go 0.10.0 among them, client-go and the go-sdk unchanged), the Bitbucket Server fixture moved from 9.6 to 10.5 with basic authentication forced back on, which the import scenarios use, the copyright and SPDX header in every source file, which is comments, and the access level and edition tier stack: the access levels each write offers, cut to what its route accepts, a pending access request no longer reported as approved, the two checksum filters gone from the project lists and new_epic from a project's notification settings, and scenarios of its own in accessrequests_test.go and accesstokens_test.go. The plaintext listener warning of #1561 is HTTP mode only, and the e2e suites drive stdio. None of it touches the two token lists, their packages or the scenarios this stack adds.

#1517

#1517 is open, and the maintainer has asked for changes before merging it. This stack leaves to it everything it does or was asked to do: the offset paging of the merge request diffs in mr_review.changes_get, with the pagination block its output gains and without keyset continuation, the page walk of the inline comment pre-check under RQB-013 with its check of the position's SHAs, and the curated text, request declarations and test moves the review asked for. R-PAGE classifies mr_review.changes_get with the rest, among the 182 actions offering page_token on an offset route, no declaration of it names that action, and nothing here changes it. Its unused order_by, sort, pagination and page_token are #1530 and the pre-check's position_type is #1531, both after #1517. The two share no hand-written file: this stack changes cmd/audit_1to1/internal/structs/analyze.go, the groupserviceaccounts and projectserviceaccounts packages, R-PAGE in cmd/audit_1to1/internal/paths (analyze.go, pagination*.go, shapes.go) and the live record's generator, and #1517 changes sent_declarations.go in the same package, the mr* packages, internal/toolutil/diff_position.go and internal/tenancy, so a rebase of either is mechanical. Both change generated files: internal/tools/testdata/tools_individual.json, tools_meta_compact.json and tools_meta_full.json, docs/development/request-inventory.json, docs/development/token-footprint.md and site/src/data/token-footprint.json with the README token block, docs/development/testing/testing.md, llms-full.txt and llms-full-individual-tools.txt. Whichever of #1517 and this stack lands second regenerates those files on top of the other, together with the ones only #1517 changes and this stack regenerates unchanged (action-requests.json, the fine-grained table, tools_meta.json, llms-full-meta-tools.txt and the mr-review tool reference).